Persistent aerial video surveillance using a wide field-of-view sensor system provides new opportunities and challenges to simultaneously monitor vehicle activities and traffic behavior in a city-size region. For instance, the low frame rate (e.g., 1-2 Hz), low spatial resolution (e.g., with ground sampling distance of about 50 cm/pixel), and sheer number of moving objects (e.g., thousands of vehicles in a single frame) make tracking and detection using wide field-of-view aerial data challenging. Moreover, the traffic models used in conventional tracking applications are primarily kinematic-constraint driven applications with strong appearance cues, and do not exploit information about road structure, derivable traffic behavior, or vehicle interaction dynamics (e.g., collision constraints). Such applications therefore tend to lose tracked vehicles in dense traffic urban environments.